Today, Internet services are delivering a large array of business, government, and personal services. Similarly, mission critical operations, related to scientific instrumentation, military operations, and health services, are making increasing use of the Internet for delivering information and distributed coordination. For example, many users are accessing the Internet seeking such services as personal shopping, airline reservations, rental car reservations, hotel reservations, on-line auctions, on-line banking, stock market trading, as well as many other services being offered via the Internet. Many companies are providing such services via the Internet, and are therefore beginning to compete in this forum. Accordingly, it is important for such service providers (sometimes referred to as “content providers”) to provide high-quality services.
One measure of the quality of service provided by service providers is the end-to-end performance characteristic. The end-to-end performance concern of service providers. In general, the end-to-end performance perceived by a client is a measurement of the time from when the client requests a service (e.g., a web page) from a service provider to the time when the client fully receives the requested service. For instance, if a client requests to access a service provided by a service provider, and it takes several minutes for the service to be downloaded from the service provider to the client, the client may consider the quality of the service as being poor because of its long download time. In fact, the client may be too impatient to wait for the service to fully load and may instead attempt to obtain the service from another provider. Currently, most website providers set a target client-perceived end-to-end time of less than six seconds for their web pages. That is, website providers typically like to provide their requested web pages to a client in less than six seconds from the time the client requests the page.
A popular client-server network is the Internet. The Internet is a packet-switched network, which means that when information is sent across the Internet from one computer to another, the data is broken into small packets. A series of switches called routers send each packet across the network individually. After all of the packets arrive at the receiving computer, they are recombined into their original, unified form. TCP/IP is a protocol commonly used for communicating the packets of data. In TCP/IP, two protocols do the work of breaking the data into packets, routing the packets across the Internet, and then recombining them on the other end: 1) the Internet Protocol (IP), which routes the data, and 2) the Transmission Control Protocol (TCP), which breaks the data into packets and recombines them on the computer that receives the information. TCP/IP is well known in the existing art, and therefore is not described in further detail herein.
One popular part of the Internet is the World Wide Web (which may be referred to herein simply as the “web”). Computers (or “servers”) that provide information on the web are typically called “websites.” Services offered by service providers' websites are obtained by clients via the web by downloading web pages from such websites to a browser executing on the client. For example, a user may use a computer (e.g., personal computer, laptop computer, workstation, personal digital assistant, cellular telephone, or other processor-based device capable of accessing the Internet) to access the Internet (e.g., via a conventional modem). A browser, such as NETSCAPE NAVIGATOR developed by NETSCAPE, INC. and MICROSOFT INTERNET EXPLORER developed by MICROSOFT CORPORATION, as examples, may be executing on the user's computer to enable a user to input information requesting to access a particular website and to output information (e.g., web pages) received from an accessed website.
In general, a web page is typically composed of a mark-up language file, such as a HyperText Mark-up Language (HTML), Extensible Mark-up Language (XML), Handheld Device Mark-up Language (HDML), or Wireless Mark-up Language (WML) file, and several embedded objects, such as images. A browser retrieves a web page by issuing a series of HyperText Transfer Protocol (HTTP) requests for all objects. As is well known, HTTP is the underlying protocol used by the World Wide Web. The HTTP requests can be sent through one persistent TCP connection or multiple concurrent connections.
As described above, service providers often desire to have an understanding of their end-to-end performance characteristics. Effectively monitoring and characterizing the end-to-end behavior of web transactions is important for evaluating and/or improving the web site performance and selecting the proper web site architecture for a service provider to implement. Because in this forum the client-perceived website responses are downloaded web pages, the performance related to web page downloading is one of the critical elements in evaluating end-to-end performance. However, the nature of the Internet and the manner in which services are provided via the web result in difficulty in acquiring meaningful performance measurements. For instance, the best effort nature of Internet data delivery, changing client and network connectivity characteristics, and the highly complex architectures of modern Internet services makes it very difficult to understand the performance characteristics of Internet services. In a competitive landscape, such understanding is critical to continually evolving and engineering Internet services to match changing demand levels and client populations.
Two popular techniques exist in the prior art for benchmarking the performance of Internet services: 1) the active probing technique, and 2) the web page instrumentation technique. The active probing technique uses machines from fixed points in the Internet to periodically request one or more Uniform Resource Locators (URLs) from a target web service, record end-to-end performance characteristics, and report a time-varying summary back to the web service. For example, in an active probing technique, artificial clients may be implemented at various fixed points (e.g., at fifty different points) within a network, and such artificial clients may periodically (e.g., once every hour or once every 15 minutes) request a particular web page from a website and measure the end-to-end performance for receiving the requested web page at the requesting artificial client. A number of companies use active probing techniques to offer measurement and testing services, including KEYNOTE SYSTEMS, INC. (see http://www.keynote.com), NETMECHANIC, INC. (see http://www.netmechanics.com), SOFTWARE RESEARCH INC. (see http://www.soft.com), and PORIVO TECHNOLOGIES, INC. (see http://www.porivo.com).
The active probing techniques are based on periodic polling of web services using a set of geographically distributed, synthetic clients. In general, only a few pages or operations can typically be tested, potentially reflecting only a fraction of all user's experience with the services of a given web service provider. Further, active probing techniques typically cannot capture the potential benefits of browser's and network caches, in some sense reflecting “worst case” performance. From another perspective, active probes comprise a different set of machines than those that actually access the service. For example, the artificial clients used for probing a website may comprise different hardware and/or different network connectivity than that of typical end users of the website. For instance, most users of a particular website may have a dial-up modem connection (e.g., using a 56 kilobyte modem) to the Internet, while the artificial clients used for probing may have direct connections, cable modem connections, Integrated Services Digital Network (ISDN) connections, or Digital Subscriber Line (DSL) connections. Thus, there may not always be correlation in the performance/reliability reported by the probing service and that experienced by actual end users.
Finally, it is difficult to determine the breakdown between network and server-side performance using active probing, making it difficult for service providers to determine where best to place their optimization efforts. That is, active probing techniques indicate the end-to-end performance measurement for a web page, but it does not indicate the amount of latency that is attributable to the web server and the amount of latency that is attributable to the network. For instance, a service provider may be unable to alter the latency caused by congestion on the network, but the service provider may be able to evaluate and improve its server's performance if much of the latency is due to the server (e.g., by decreasing the number of processes running on the server, re-designing the web page, altering the web server's architecture, etc.).
The second technique for measuring performance, the web page instrumentation technique, associates code (e.g., JAVASCRIPT) with target web pages. The code, after being downloaded into the client browser, tracks the download time for individual objects and reports performance characteristics back to the web site. That is, in this technique, instrumentation code embedded in web pages and downloaded to the client is used to record access times and report statistics back to the server. For example, a web page may be coded to include instructions that are executable to measure the download time for objects of the web page. Accordingly, when a user requests the web page, the coded instrumentation portion of the web page may first be downloaded to the client, and such instrumentation may execute to measure the time for the client receiving each of the other objects of the web page.
As an example, WEB TRANSACTION OBSERVER (WTO) from HEWLETT PACKARD's OPENVIEW suite uses JAVASCRIPT to implement such a web page instrumentation technique (see e.g., http://www.openview.hp.com/). With additional web server instrumentation and cookie techniques, this product can record the server processing time for a request, enabling a breakdown between server and network processing time. A number of other products and proposals employ similar techniques, such as the TIVOLI WEB MANAGEMENT SOLUTIONS available from IBM CORPORATION (see http://www.tivoli.com/products/demos/twsm.html), CANDLE CORPORATION'S EBUSINESS ASSURANCE (see http://www.candle.com/), and “Measuring Client-Perceived Response Times on the WWW” by R. Rajamony and M. Elnozahy at Proceedings of the Third USENIX Symposium on Internet Technologies and Systems (USITS), March 2001, San Francisco.
Because the web page instrumentation technique downloads instrumentation code to actual clients, this technique can capture end-to-end performance information from real clients, as opposed to capturing end-to-end performance information for synthetic (or “artificial”) clients, as with the above-described active probing techniques. However, the web page instrumentation technique fails to capture connection establishment times (because the instrumentation code is not downloaded to a client until after the connection has been established), which are potentially an important aspect of overall performance. Further, there is a certain amount of resistance in the industry to the web page instrumentation technique. The web page instrumentation technique requires additional server-side instrumentation and dedicated resources to actively collect performance reports from clients. For example, added instrumentation code is required to be included in a web page to be monitored, thus increasing the complexity associated with coding such web page and introducing further potential for coding errors that may be present in the web page (as well as further code maintenance that may be required for the web page).